Three-Dimensional Speaker Localization: Audio-Refined Visual Scaling Factor Estimation

Neither a monocular RGB camera nor small-size microphone array is capable of accurate three-dimensional (3D) speaker localization. By taking advantage visual object detection, and audio-visual complementary sensor fusion, we formulate the localization problem as scaling factor estimation problem. As result, effectively reduce traditional audio-only 3D from an exhaustive grid search to one-dimensional (1D) optimization We propose multi-modal perception system with two approaches. show that proposed methods are effective, accurate, robust against interference and, corroborated by indicative empirical results on real dataset, competitive conventional uni-modal state-of-the-art